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Integrated Information Systems for the International Space 


Station! 




Hazards 


y 

'CordicaUons 






OCADs 
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MADS 



Planning and Scheduling Tools 




Mobile Applications for Operations and 
Systems Engineering! 




Vibration Impact on Human Performance! 







National Aeronautics and 
Space Administration 


NASA 



. V 


Discovery Innovations Solutions 


Vision Science and Visual Technologies! 


Image Compression, Frame Rates and TBI Assessment 





Airspace Operations Laboratory! 












Human Systems Integration for UAS and 
Single Pilot Operations (or “Reduced Crew 

OnpratinriQ’^l , 





Aviation Safety Reporting System (ASRS)! 

•" Processes voluntarily submitted incident reports from pilots, controllers, flight 
attendants and others.! 

•" Reports may describe unsafe and hazardous situations.! 

V ■> 

•" Receiving about 1 0,000 reports per month. ! 

•" Established in 1976.! 


ASRS 

Database 



Google SDC Collaboration 
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NASa-Ames Research Center. 


Rational Aeronautics and Space Administration 


Human Systems 

integration division 


Modeling and simulation are critical to 
comprehensively study complex human-system 
designs 

Many different types of models exist at NASA 

Human behavioral 
Human performance 

Anthropometric, biomechanic, volumetric 
Information processing 

Vision, auditory, memory, and other human processes 
Task network 

Physical structural (space launch vehicle, aircraft, 
crewstations, CAD/CAEs) 

Airspace system 
Weather 

Airflow and other CFD 
Physiological 
Robotics and automation 
Oxygen and blood flow 
Scheduling 




yf Human Systems 

TTASAAmes Human Modeling in System Design" 


n 


BRAHMS: Agent-based 
models 

MIDAS: Behavior models 
Cognitive process models: 
Decisions / response to 
information 

SPIFe/SCORE: Scheduling 
and Planning models 


►"Airspace 
►"Aircraft Trajectories 


•'trewstation 
►'Flight deck layout 





ACES: National Airspace 
System 

FACET: Air traffic 
management 


•'Illumination 

•'Loads 

JACK: Anthropometric 

•"Gravitational forces 

•"Manual handling 
•"Fluids and heat 

model 


'ansfer models 

Biomechanic model 

System and 

Physical 

MIDAS-FAST (& BORIS): 

Environment 

Equipment 

Robotics Trainer 

Design 

Design 

Volumetric 


CFD 



• 'Procedures 

• 'Training 

• 'Poles & 
responsibilities 

• 'Scheduling 


Operations 

Design, 

Evaluation, 

and 

Integration 


Technology 

Design, 

Evaluation, 

and 

Integration 


""Displays 

•"Automation 

•'Information 

Requirements 


ADEPT: Human- 
Automation Interaction 
HOP: Human Vision 
Basic Process Models: 
Audition (e.g. threshold 
models), 

MIDAS: Memory & 
cognitive processing 
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NASA Ames HSI Research Areas 
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http://humansystems.arc.nasa.gov/techareas/tech_areas.php 


Contribute to the development of better NASA software through careful application of iterative 
user research, interaction design, and usability (Curiosity - MSLICE) 

Human Automation ADEPT provides a tool for prototyping automation and associated interfaces, in an integrated tool 

Interaction that includes analyses to identify potential HAI vulnerabilities early in the design process 


Laboratory 


HCI 


■ 


Research Area 


Scheduling and Planning 

Scheduling and Planning Interface for Exploration (SPIFe) toolkit for space missions that includes 
human constraints on mission operations 

Advanced Controls Ld - 
and Displays 

Research on haptics, speech recognition, visual perception, visual perception in space, adaptation 
to virtual environments, and acoustics 

Human Vibration Laboratory 

Assesses whole-body vibration impacts on visual, cognitive, and manual performance, 
understand the mechanisms contributing to vibration-induced performance deficits, and develop 
countermeasures to mitigate these deficits. 

Flight Cognition Lab 

Studies the cognitive, team and organizational processes that underlie the performance of pilots, 
air traffic controllers, and other skilled professionals 

Psychophysiological Lab 

Studies altered gravitational effects on human autonomic and central nervous system function to 
maximize the health, productivity and safety of humans in space. 

Intelligent Systems 

Enhance mission safety and crew efficiency in next-generation spacecraft by evaluating the 
operational impacts of environmental stressors and by developing and testing advanced 
operations concepts and crew-vehicle interfaces. 

Man-machine Integration 
Design and Analysis System 
(MIDAS) 

Develop human performance models of human-system interaction to predict operator 
performance along the measures of task performance and times, visual attention, workload, 
situation awareness 

Airspace Operations 
Laboratory (AOL) 

Researches roles, responsibilities, and requirements for human operators and automation in 
future air traffic management (ATM) systems using human in the loop 

Human Centered Systems 
Laboratory (HCSL) 

Focuses on mission safety and efficiency by developing innovative display technologies using both 
HITLand HPM methodologies 

Flight deck display research 

15 

works to increase the capabilities of the flightdeck crew by expanding their roles and 
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Human Performance Models (HPMs) 
allow system designers the ability to 
model critical events that cannot be 
fully studied with empirical simulations 


Models can be used to provide 
estimates of human-system 
performance when the concepts, 
technologies, or automation are too 
new, difficult, or dangerous for the 
human operator 


Model validity is a paramount concern 
when predictions are generated to 
evaluate candidate NextGen 
operations 
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Motivation: 
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integration division 


NextGen Technology Design, Evaluation, and Integration 
•" NextGen Characteristics: 


More data available to the flight deck 

•" e.g., weather, wake, traffic trajectory projections, etc. 

More precise and closely coordinated operations 

•" e.g., self-separation, closely spaced parallel operations, RNAV/RNP 

More tasks are automated 

•" Pilot increasingly placed in a monitoring role 

Potential for increased workload, decreased situation 

awareness, increased demand for shared attentional resources 


" Evaluating NextGen Concepts: 



Must consider pilots’ capabilities when designing / evaluating 
NextGen procedures, operations, roles / responsibilities and 
the information requirements 

Failure to do so will leave the pilots, and thus the entire aviation 
system, vulnerable to error 

MIDAS 
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Hume 

Iwart^machine Integration Design and Analysis System (MIDAS) 


Human Systems 

ation division 


S" Validated, first-principle models of human 
behavior including perception, visual attention, 
memory, & workload 

S" 3D CAD models of the environment, the 
workstation, and the equipment 

S" Controls a generic, anthropometrically-correct 
human mannequin (Jack™, 5th percentile female 
- 95th percentile male) 

S" Monte carlo simulation capability with stochastic 
human performance 




v'" Distributed simulation (e.g. Microsaint Sharp) 

S" Generates realistic task-management 
behaviors sensitive to task context, 
environment 

v'" Produces task timelines, workload, and 
situation awareness profiles and visualization 
which permits testing of procedure alternatives 


man-machine integration design 
and analysis system 
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MIDAS Input 


i 


Tasks and Procedure Lists 
(activities and sub-activities) 


Microsaint Sharp 


ission Models 
Workstation Models 
Anthropometric Models 
Environmental Models 
Dynamic Models 
Flight Profile Models 
Scenario Objects 


MIDAS v5 Structures 
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Schedules 
Actuates/Triqqers 


Comr 

nands 

A 

k 

1 

r 

Res 

ults 



Physical Simulation 

Environmental behavior 
Crewstation behavior 
Model state movement 
Model state actions 
Model state changes 


1 


; Library 

Primitive tasks in hur 

i 

nan model 


Operator Characteristics 
Performance Shaping Factors 



MtDAS 



MIDAS Operator Process Models 

Fitts Law; Perception & Attention (SEEV) 
Multiple Resource Model; Memory, SA, 
Workload; Operator States (fatigue, 
gravitational effects); Timeliness 
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\\ [] | Timeline 
\\ [] ^it/Reach/Vis envel ope 
[] [] | Dynamic Animatio n 




0 G I Mission Risks 


G 1 Mission success 


P G Nrf° rmance measures 



Workload, visual 
attention 

HE 
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CSPO Project Overview 


•" Objective 


Develop valid HPMs of approach and land operations, use these models to 
evaluate candidate NextGen concepts (Closely Space Parallel Operations, 
CSPO), develop guidelines regarding flight deck displays and pilot roles and 
responsibilities 



r \ 

•'RNAV 
Approach and 
Land 


" ' \ 

•'2 Operational 
scenarios of 
CSPO concepts 



•'Compare CSPO 
workload, 
visual fixations 
to RNAV 

J 
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Gore, B.F., Hooey, B.L., & Foyle, D.C. (BRIMS 2011, March 21-26). 


"What-if" 

Investigations 



•'Off-Nominals 

•'Roles, 

Responsibilities and 
Flight Deck Displays 
( 2011 ) 


Validate 


Evaluating NextGen Closely Spaced Parallel Operations 
Concepts with Validated Human Performance Models 

Model Development and Validation 

Develop RNAV model based on task analyses (SME input) 
Validate model inputs, processes and outputs 
Extend RNAV model to two CSPO Concepts 
Evaluate : Pilot-ATC separation responsibility. 

Wake Information Requirements, 

Spacing Management Information Requirements 
Implications based on: Pilot workload, visual attention 
event/alert detection response times 



MIDAS v5, a human performance 
model of the flight deck environment, 
pilots' workflow and cognitive 
processes. 


CSPO 

200 ' 


CSPO Approach: 200 ' Ceiling 

Autoland 

Paired 

with Lead ^ 

^ Speed-jCoupled ^ 

^ Parallel \y\th Leai 

i i /■ — • — 4 i 


w 1 

~>f VMC 

1 DH 1 


Altitude: 10,000 4000 3000 1800 1100 800 650 200 0 

Phase: Descent Approach Land TD 






RNAV scenario + 2 CSPO 
operational scenarios 
-"Validated model 

- inputs (Focus groups) 

- processes (Literature) 

- outputs (HITL data) 

- Weather (high wind) 

- RNP Loss 

- FMS Failure 

- Aircraft of runway 

-"Pilot-pilot roles 
(Allocation of task, 
monitoring workload) 
-"Pilot-ATC roles 
(Conflict detection and 
resolution) 

-"Flight deck information 
required to support early 
conflict detection and safe 
response 

-"Wake format and location 
-"Spacing Automation style 
and format 
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•" Insert MIDAS TOGA movie 
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— “ CSPO Findings and Implication ^* 1 

Flight Deck Requirements for: 

1. " ATC-Pilot Roles and responsibilities: ATC vs Pilot responsibility for separation 

2. " Alert styles for wake and blunder threats: One-stage vs two-stage alerts 

3. " Wake display technology: Format (predicted vs real-time), Location (PFD, Nav Display, or Both) 

4. " Spacing management automation: Style (Current vs NextGen), Location (PFD, Nav Display, or Both) 

Evaluating ATC-Pilot Roles and Responsibility: Separation Delegation 

-" Compared Current-day (ATC responsible for separation) with NextGen (Pilot responsible for separation) 
-" Model predicted slightly faster emergency escape maneuvers when Pilot's are responsible (.3 sec), BUT... 


Higher workload when pilots are 

Less balanced pilot scan when pilots are 

responsible for separation 

responsible for separation 



PFD 


NAV OTW 23 


Hooey, Gore, Mahlstedt, & Foyle, (2012); Gore, Hooey, Mahlstedt, Foyle (in process) 


CSPO Findings and Implications 

Flight Deck Requirements for: 

1. " ATC-Pilot Roles and responsibilities: ATC vs Pilot responsibility for separation 

2. " Alert styles for wake and blunder threats: One-stage vs two-stage alerts 

3. " Wake display technology: Format (predicted vs real-time), Location (PFD, Nav Display, or Both) 

4. " Spacing management automation: Style (Current vs NextGen), Location (PFD, Nav Display, or Both) 

Flight deck requirements for spacing management automation style 

-" Compared Current-day spacing management (MCP) with NextGen Automation (e.g. Airborne Spacing for 
Terminal Arrival Routes (ASTAR) algorithm; Murdoch, 2009) 

-" Extend Lozito et al. HITL results to assess pilot scan and response to off-nominal events 


Less time monitoring spacing with I Faster to detect RNP-Loss I Slower to detect Automation 
NextGen Automation I (EICAS) with NextGen I Failure (PFD) with NextGen 



Current-day speed-management resulted in: increased pilot scans to spacing displays, faster 
detection of RNP-loss alert (on EICAS) 

NextGen speed-management slowed time to detect automation failure on PFD (complacency) 

Hooey, Gore, Mahlstedt, & Foyle (2012) 
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SPO Project Overview 


•" Objective 

Complete a task analysis of the Pre-TOD to TOD tasks required in candidate 
Operational scenarios (Single Pilot Operations; SPOs) 


Develop 
Current Day 
Model 


•RNAV 

Approach and 
Land 



scenarios of 
CSPO concepts 




workload, 
visual fixations 
to RNAV 

J 




u 



MIDAS 

man-machine integration design 
and analysis system 


Nominal Scenario 
Off Nominal Scenario Off Nominal Scenario 




V&V tasks & 
operations 
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Denver Arrival Approach Plate 


•" Nominal Approach plate to DIA 


•" Off-Nominal (Divert Approach 
plate to Cheyenne) 
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2013 

4 entities 



2014 

9 entities 



Pilot on Board 
Ground Operator 
Automation 
ATC 



4 Scenarios 

Current Day 
•" Nominal 
•" Off-Nominal 
SPO 

•" Nominal 
•" Off-Nominal 
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Onboard Pilot 
FD Automation 
Ground Operator 1 
Ground Automation 1 
Ground Operator 2 
Ground Automation 2 
Ground Operator 3 
Ground Automation 3 
ATC 


5 Scenarios 


H 


H 


Current Day 
•" Nominal 
•" Off-Nominal 
SPO High Build-up 
•" Nominal 
•" Off-Nominal 
SPO No Build-up 
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Generating the 2014 Task Analysis 


Human Systems 
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•" Modified a verified set of flying tasks completed 
•" No FO/Limited FO 

•" New dispatch tasks added (including handoffs) 
•" Redistribution of tasks among 9 operators 
•"All tasks must be assigned 
•" Greater delegation to automation 

-'When tasks were moved to automation, new 
crosschecking tasks arose for the human operators 

MIDAS 

man-machine integration design 
and analysis system 
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•" Focused on defining task groups (for flexibility) 

Dispatch 

•" Scans for off-nominal situations 
•" Addresses maintenance issues 
•" Reroutes 

•" Complies to Company standards 
•" Liaison between OBP and outside entities 

-"OBP 

•" Flying tasks (addresses clearances, executes clearances, communicates w/ 
ATC & GO) 

•" Continuous Tasks- Crosschecks flight against CA’s mental map 

-"Automation 

•" Ground- Notifications: flight conformance, off-nominals, and reminders to act 
or check. Collect and organize flight information (handoff packages). 

•" FD- Notifications, Reminders, and Loads of clearances 

•" Communicate between Ground & FD Automations 

MIDAS 

man-machine integration design 
and analysis system 
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•" Included: 


Nominal “shift change” handoff as 
one of the first task groups 

Nominal ramp-up 
Off-Nominal handoff 
Off-Nominal ramp-up 
“End-of-DA” handoff 
“End-of-DA ramp-up 
Dispatch tasks 
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•" Discovered: 

Greater reliance on automation 


Ground Automation (new) 
“Dispatch” Automation (new) 

New Relationships 

•" GO > GO, GO > FD, FO > GO, 
OBP > GO, CA> FO 

What happens during a DA 
handoff? 


Who is responsible for the 
dispatch tasks? 

Should a Ground FO in DA have 
assigned duties, or are they 
determined by the CA> FO 


relationshi 
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Summary 

1 HPMs such as MIDAS can be used to evaluate: 

-"Pilot/ATC tasks, roles and responsibilities, and function allocation 
-"Technology development and integration 
Error or safety vulnerabilities 
-"Procedures and training needs 

2." HPMs and the modeling approach can be applied to other: 

-"Phases of flight, (e.g. aviation - arrivals, enroute, departures, taxi and 
their transitions; space - ascent, descent; ISS operations) 

— " Flight deck technologies (e.g., SVS/EVS; CDTI; EFBs; MFDs; ) 

-" Information requirements manipulations 

-"Concept of Operations evaluations (e.g. Single Pilot Operations) 

-"Space operations (e.g. ISS and CEV/SLS procedure design and 
evaluation) 

-"Human-automation interaction domains 
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